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Abstract 

We consider continuous-time Markov kinetics with a finite number of states 
and a positive equilibrium P* . This class of systems is significantly wider than 
the systems with detailed balance. Nevertheless, we demonstrate that for an 
arbitrary probability distribution P and a general system there exists a system 
with detailed balance and the same equilibrium that has the same velocity dP/ dt 
at point P. The results are extended to nonlinear systems with the generalized 
mass action law. 
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1. Introduction 

1.1. Detailed balance and beyond 

The principle of detailed balance is one of the most celebrated results in 
kinetics. A kinetic system is represented as a mixture of independent elementary 
processes (collisions or elementary reactions, for example). Due to the principle 
of detailed balance, at equilibrium, each elementary process should be equilibrated 
by its reverse process. Kinetics is decomposed into pairs of mutually inverse 
processes and in many problems we can consider these pairs separately. 

We study relations between the systems with and without detailed balance. 
In this Section, we briefly overview the main results of the work. Then, in 
Sec. 11.21 we review the history of the problem. We prove the local equivalence 
theorem for the Markov processes in Sec. [2] and give there the simple examples. 
The nonlinear generalizations are presented in Sec. [3] 

In Sec. [2] we start from the first order kinetics without the detailed balance 
assumption. The general first order kinetic equation has the form: 
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where = 1, ...,n, i ^ j) are non-negative. This system of equa- 

tions (master equations or Kolmogorov's equations) describes dynamics of non- 
negative variables pi (i = 1, ...,n). These variables may be considered as 
probabilities (then '^2 i Pi = 1) or concentrations. For the corresponding states 
or components we use the notation Ai. In this notation, g,y is the rate constant 
for transitions Aj —> A,-. 

Let us assume that system {1} has a positive equilibrium P* = (p*), p* > 0: 

^2 q ^ p j = X! q i l p * for alH = 1, . . . n . (2) 

This is the so-called balance equation. 

The detailed balance condition is much stronger. It assumes that the sums 
in the left and right hand sides of Eq. ([2]) are equal term by term: 

HjPj = QjiPi for B&i,j = l,...n,i^j . (3) 

For the number of states n > 2, a simple comparison of dimensions demon- 
strates that there are much more general systems with the given positive equi- 
librium P* ([2]) (dimension is (n — l) 2 ) than the systems with detailed balance 
with equilibrium P* (dimension is " 2 ~ )■ Surprisingly, for every given 
distribution P, the set of possible velocities dP/dt for general Markov kinetics 
with equilibrium P* is the same that for Markov kinetics with detailed balance 
and the same equilibrium. This is the central result of the paper (Theorem [T] in 
Sec. 12}. 

We demonstrate this in two steps. First, we use the representation of a 
general Markov chain with a given positive equilibrium as a combination with 
non-negative coefficients of several simple cycles with the same equilibrium. 

Secondly, we demonstrate this equivalence for a simple cycle of transitions 
with positive constants 

Ax -> A 2 -> . . . -> A n -> Ax . (4) 

For the equilibrium P* the constants of the cycle are qi+u = n/p* (we use here 
the standard convention about the cyclic numeration, n + 1 = 1). 

Thus, if we observe the Markov kinetics at one point then we can not dis- 
tinguish general systems from systems with detailed balance because the sets of 
possible velocities coincide. In particular, they have the same set of Lyapunov 
functions. 

Our main results allow us to decompose any Markov kinetics (or generalized 
mass action law kinetics with semi-detailed balance) into pairs of mutually in- 
verse elementary processes with the same equilibrium. If the system does not 
satisfy the principle of detailed balance then this decomposition depends on the 
state. Nevertheless, in some problems it is still convenient to consider these 
pairs separately. 

In this paper, we give two examples of the application of Theorem [TJ the 
evaluation of logarithmic decrement for general Markov chains and a simple 
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Figure 1: Elementary reaction with intermediate compounds. 



proof of the Morimoto iJ-theorem for all the Csiszar-Morimoto divergencies. 
We give also the nonlinear generalization of Theorem [T] for the systems which 
obey the generalized Mass Action Law (MAL). 

Master equation is a source for many other kinetic equations. In particular, 
in Sec. |3]we consider complex reactions with intermediate compounds (Fig. [IJ 
under two asymptotic conditions 

• The compounds Bj are in fast equilibrium with the corresponding input 
or output reagents; 

• They exist in very small concentrations compared to other components. 

For compounds transitions the first order kinetics is assumed because of small 
concentrations of compounds (only first order terms survive). These assump- 
tions allow us to produce the reaction rates for the overall reaction from Fig. Q] 
in the form of the generalized MAL: 

-),¥>, >Q- (5) 

where /ii is the chemical potential of the component A4, pis the reaction number, 
a P i are the input stoichiometric coefficients (Fig. [1} . Both a P i and (3 p i are non- 
negative integers. We use notations ot p and (3 p for vectors wit coordinates 
a pi and j3 p i. The positive functions ip p are called the kinetic factors whereas 
exp(J2i cvpifJ'i/ RT) are the Boltzmann factors. 

The balance condition of the first order kinetics of compounds @ transforms 
in the semi-detailed balance condition (that is known also as the complex or the 
cyclic balance condition): 

p, a p =v p,P p =v 

for any vector u from the set of all vectors {a p ,f3 p }. Kinetics with the general- 
ized MAL and the semi-detailed balance conditions give the natural nonlinear 
generalizations of Markov processes. In particular, the entropy production for 
these systems at any nonequilibrium state is positive. 

If we assume for the Markov kinetics of compounds that the positive equi- 
librium is the point of detailed balance ([3]) then the kinetic factors tp p satisfy 
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the stronger condition of detailed balance: 

ft = for a11 P ' 

where tp+ is the kinetic factor for the direct reaction and ip~ is the kinetic factor 
for the reverse reaction. 

The class of systems with semi-detailed balance is much wider than the class 
of systems with detailed balance. Nevertheless, locally they coincide: the set of 
possible velocities for systems with semi- detailed balance coincide with the set of 
possible velocities for the systems with detailed balance for given thermodynamic 
functions and any given state (Sec. [3]). 

The systems with generalized MAL and semi-detailed balance are the nonlin- 
ear analogs of the Markov processes and the local equivalence of the generalized 
MAL systems with detailed and semi-detailed balance is the analog and a con- 
sequence of Theorem [TJ 

1.2. A bit of history 

In 1872, Boltzmann introduced the principle of detailed balance for collisions 
and used it to prove his iJ-theorem Boltzmann's proof of the positivity 
of entropy production for systems with detailed balance is very transparent 
because it is sufficient to prove this positivity just for a couple of mutually 
inverse elementary processes. 

In 1887, Lorentz Q objected Boltzmann: he insisted that some collisions 
of polyatomic molecules do not have reverse collisions and cannot satisfy the 
principle of detailed balance. Immediately, Boltzmann realized that there ex- 
ists a much weaker condition sufficient for the _ff-theorem 3\. In 1981, it was 
proven that the Lorentz objections are wrong and the principle of detailed bal- 
ance is valid for polyatomic molecules |4j. This is not very surprising because 
the detailed balance follows from microreversibility (or T-invariance of the fun- 
damental equations of mechanics or quantum mechanics). Nevertheless, the 
Boltzmann discovery is valuable by itself and is used for many kinetic equa- 
tions. 

This condition was rediscovered several times. It is known as the semi- 
detailed balance condition, the cyclic balance condition or the complex balance 
condition. In 1952, Stueckelberg proposed a proof of the semi-detailed balance 
condition for the Boltzmann equation [5J. His proof is based on the Markov 
model of elementary events. Recently, the Stueckelberg approach was extended 
to prove the semi-detailed balance condition for the generalized MAL kinetics 
0- 

The complex balance condition for chemical kinetics was introduced by Horn 
and Jackson in 1972 7] independently of Boltzmann's work. Now it is used for 
mathematical modeling in chemical kinetics and engineering Q. Boltzmann's 
idea about cyclic balance developed in physical kinetics was independently re- 
discovered in the theory of Markov processes and it is proved that any recurrent 
Markov process can be decomposed into directed cycles [loj . 
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The principle of detailed balance was crucially important in the development 
of the Metropolis-Hastings and other Markov chain Monte Carlo algorithms 
from the very beginning [11]. Technically, it is much easier to use the detailed 
balance conditions © than to follow more intricate balance conditions. Detailed 
balance was considered as a necessary condition for construction of Monte Carlo 
algorithms as a "systematic design principle" 12]. It was demonstrated that it 
helps to reduce the uncertainty of some observables in stochastic numerics 13 1. 

Nevertheless, there are many examples of efficient Monte Carlo computa- 
tions without detailed balance. Sometimes computational models without de- 
tailed balance are constructed because of the physical nature of the systems. For 
example, the models of inelastic processes in particle physics 14| or in granular 
media [15[ may violate the principle of detailed balance. The general theories of 
stochastic cellular automata with Gibbsian equilibria but without compulsory 
detailed balance were developed 16]. It is widely recognized that the balance 
equation (not the detailed balance) is necessary and sufficient condition for in- 
variance (stationarity) of the desired equilibrium distribution. Under some more 
technical irreducibility conditions, the Monte Carlo simulations will converge to 
this equilibrium (l7lll8j|. 

The interplay between reversible (with detailed balance) and irreversible 
Markov chains is non-trivial and important for many applications. Recently, 
it was demonstrated that the local deformation of the reversible Markov pro- 
cesses into irreversible ones helps to create efficient computational Monte Carlo 
algorithms [l9j |. 

Much efforts were applied to verification of microscopic reversibility and 
its consequences, the detailed balance conditions and the Onsager reciprocal 
relations, in many experimental systems IS 21. 22, 23. 24 \. To check experi- 



mentally the detailed balance conditions it is necessary to deal with a complex 
reaction that can be formally equilibrated without detailed balance. If not, then 
one tests not the detailed balance but just the equilibrium condition as it was 
mentioned in [25j |. 

The detailed balance conditions are very natural and appealing. They sim- 
plify many computations and proofs. Thus, they are used in many applications 
and models. For some physical and chemical systems, detailed balance has a 
solid background, the T-invariance of the fundamental equations, but it is also 
used beyond the proven microreversibility. When modelling with detailed bal- 
ance meets some difficulty then the problem about relations between reversible 
and irreversible systems arises again and detailed balance is substituted by more 
general conditions. Nevertheless, the convenience, beauty and some intrinsic 
benefits of the phenomenon, have forced researchers to return to detailed bal- 
ance if it is possible without a conflict with reality. There are many examples 
of this "pendulum" in scientific publications: accept detailed balance - criticize 
detailed balance - go to more general conditions - realize the benefits of detailed 
balance - return to detailed balance - ... 

At the same time, the consequences of the principle of detailed balance are 
extended to the situations where it was not used before. Thus, recently the 
multiscalc limit of the systems of reversible reactions was studied when some of 
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the equilibrium concentrations tend to zero. The extended principle of detailed 
balance was proved for the systems with some irreversible reaction [26|, [27} ■ 

In Theorem [TJ we compare the sets of possible velocities, dP/dt, for two 
classes of systems: (i) general first order kinetics (TTJ with the given positive 
equilibrium P* and (ii) first order kinetics with detailed balance © and the 
same positive equilibrium. 

Understanding the structure of the sets of possible velocities can provide 
additional information about attainable states of the system which is helpful 
in the modelling context. There are many other reasons too. It is known that 
different types of kinetics data bear different degrees of reliability. It would 
therefore be very attractive to study the consequences of the information of each 
level of reliability separately [28}. For example, uncertainty about equilibria in 
the system is usually significantly lower than that of the reaction rate constants. 
The value of the equilibrium gives us some information about dynamics: the set 
of possible velocities dP/dt is not arbitrarily wide at a given state and for given 
equilibrium. For the systems with detailed balance, this set is a polyhedral 
cone which allows a simple description (Sec. I2.2[) . Due to Theorem [TJ however, 
this cone is also the set of possible velocities for the general master equation. 
Therefore, to distinguish the detailed balance systems from the general ones we 
have to involve data about dP/ dt for several significantly distant distributions 
P. 

Another example is to employ the knowledge of the sets of possible velocities 
for estimating attainable regions for kinetics. The idea to use the equilibrium 
information to estimate the attainable sets in kinetics was proposed in 1964 by 
_and developed further in chemical kinetics and chemical engineering 
33, [HI (for more detailed review see [Hj]). If the sets of possi- 
ble velocities coincide then the estimated attainable regions coincide too. The 
knowledge of the sets of possible velocities is also important for the analysis of 
observability, identifiability and controllability of the systems. 




2. Local equivalence of general Markov systems and systems with 
detailed balance 

2.1. Global decomposition into cycles, local decomposition into steps, and the 
equivalence theorem 

Let us start from master equation (TTJ . The coefficient qij is the rate constant 
for transitions Aj — > Ai. Any set of non-negative coefficients qij (i ^ j) corre- 
sponds to a master equation. Therefore, the set of all master equations (UJ may 
be considered as the non-neg ative orthant in M" ( " 1} C M™(" _1 ^. (The non- 
negative orthant is the set of all vectors with only non-negative components.) 

We assume that system (fTJ has a positive equilibrium P* = (p*), p* > 
and the balance condition ([2j holds. The sum of these n balance conditions is 
a trivial identity. Let us delete any single equation from ©, for example, the 
last one (for i = n). Each of the remaining equations includes the variable qi n 
which is not present in other equations (i = 1, . . . , n — 1). Therefore, for given 
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positive P* , there are n—1 independent conditions on q^ (i, j = 1, . . . , n, i ^= j) 
in (O. 

Thus, the balance conditions © define for a given positive equilibrium a 
(n — l) 2 -dimensional linear subspace L* in the n(n — 1) dimensional space of 
lij (* j)- A vector of positive coefficients q*. = 1/p* satisfies ^ and belongs 
to L*. This vector belongs to the interior of the non-negative orthant IR™^™ . 
Therefore, the intersection R™ 1 -™ ^ n L* includes a vicinity of q*^ in L* and is 



a (n — l) 2 -dimensional cone. Thus, the non-negative solutions of ([2]) form a 
(n — l) 2 -dimensional closed cone in M™^™ 1 . 

The systems with detailed balance ([3]) for a given positive equilibrium form 
a smaller cone. Under these conditions, there are only "C" -1 ) independent 
coefficients among n(n — 1) numbers q^. For example, we can arbitrarily select 
Qij > for i > j and then take q^ = qjA for i < j. So, for given P*, the cone 
of the detailed balance systems ((SJ can be considered as a non-negative orthant 
in R - ^ embedded in 

If the balance condition ([2j holds then system ([1]) may be rewritten in a 
convenient equivalent form: 



(6) 



With this form of master equation, it is straightforward to calculate the time 
derivative of the quadratic divergence, a weighted 1% distance between P and 

P*,H 2 (P\\P*) = Z l ^jf 1 -- 

dH 2 (P\\P*) _ ^ J Pi p/ 




di \i>; P ' 3/ 

This time derivative is strictly negative if for a transition ^ — > ^ the rate 
constant is positive, q^ > 0, and ^ £i. Hence, if the state P is not an 

equilibrium (i.e., the right hand side in ^ is not zero) then dHl ^ P - < 0. 
Let us introduce the following notation for a given number of states n: 

• Qb(-P*) i s the cone °f the vectors of non-negative coefficients q^ (i ^ j) 
which satisfy the balance conditions ((2|), that is, the set of all Markov 
processes with the equilibrium distribution P*; 

• 2db(-P*) i s the cone of the vectors of non-negative coefficients q^ (i ^ j) 
which satisfy the detailed balance conditions ([3|), that is, the set of all 

Markov processes with detailed balance and the equilibrium distribution 

p* 

If a system satisfies the detailed balance condition ([3]) then the balance con- 
dition ([2]) holds too: it holds term by term, even without summation. Therefore, 
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Q BB (P*) C Qg(P*). Moreover, comparing dimension we find that this inclu- 
sion is strong. Indeed, dimQg(P*) = (n-1) 2 , dim Qg B (P*) = " (n ~ 1) . If n > 2 
then "^" 2 ~ 1 - ) < (n — l) 2 , hence, 

QS b (p*)cq«(p*)c m ;("-i). (8) 

The cone of the systems with detailed balance is, in some sense, much smaller 
than the cone of the systems with the balance condition: the difference between 

• • • (n-l)(n-2) 

tncir dimensions is -. 

Now, let us consider the right hand side vector fields of the systems (|TJ) at 
the point P ^ P* . For each cone of the coefficients <jry the vectors of the possible 
velocities, dP/di, also form a cone. Let us introduce the following notation: 

• Qg(P, P*) is the cone of the possible velocities, dP/di, at the point P for 

( qij ) G Qg(P*); 

• Qdb(-P> -P*) i s the cone of of the possible velocities, dP/di, at point P for 
( qij ) G QS B (P*). 

Qg (P, P* ) is the cone of all possible velocities for Markov kinetics at the point 
P if the equilibrium is P* . Q BB (P, P* ) is the cone of these velocities for Markov 
kinetics with detailed balance. 

Surprisingly, for every given distribution P, the set of possible velocities 
dP/dt for general Markov kinetics with equilibrium P* is the same that for 
Markov kinetics with detailed balance and the same equilibrium. The following 
theorem is the central result of this work. 

Theorem 1. Qg(P,P*) = Qg B (P,P*) 

This means that for every first order kinetic equation ([1]) with a given positive 
equilibrium P* and for every point P ^ P* there exists a first order kinetic 
equation with detailed balance and equilibrium P* that has the same velocity 
at P. At this point the right hand sides of the kinetic equations coincide. 

Therefore, if we observe the Markov kinetics at one point then we can never 
distinguish general systems from systems with detailed balance. In particular, 
they have the same set of Lyapunov functions: 

Corollary 1. If for a function H(P,P*), dH/dt < for any system |7P with 
equilibrium P* and detailed balance then dH/dt < for any system (QJ) with 
equilibrium P* . 

The system with detailed balance has n(n — l)/2 dimensions available to 
match a single rt-dimensional velocity vector. Therefore, it is not surprising 
that the cones Qg. B (P, P*) has non-empty interior (solid cones). But this is not 
enough to cover any n-dimensional velocity vector using non-negative coefficients 
qtj. This non-negativity condition defines the borders of the cones and we can 
a priori just state the inclusion Q BB (P, P*) C Qg(P, P*). The calculation of 
dimension does not give a hint about coincidence of these cones. 
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The proof of Theorem 1 is constructed in two steps. First, we prove that for 
every P* and P the cone of possible velocities Qg (P, P* ) is the convex hull of 
the velocities at point P of the simple cyclic schemes, Ai k — > Ai 1 

(fc < n and all the numbers . are different), with the same equilibrium 
P* . Secondly, we prove that it is sufficient to take k — 2. 

We will characterize Qg(P, P*) by its extreme rays. A ray with direction 
vector x ^ is a set {Ax} (A > 0). I is an extreme ray of a cone Q if for any 
u € 2 and any x,y S Q, whenever it = (x + y)/2, we must have x,y £ I. If a 
closed convex cone does not include a whole straight line then it is the convex 
hull of its extreme rays [35j . 

Lemma 1. The cone Qg(P, P*) does not include a whole straight line. 

Proof. If v 7^ is a possible value of the right hand side of (O then the derivative 
of H2(P\\P*) in direction v is strictly negative ([7]). Therefore, it is impossible 
that both v and -v belong to Qg(P, P*). □ 

Let us consider a simple cyclic scheme, A„ A ifc — > (k < n and 

all the numbers are different). For a given positive equilibrium, P* 

the coefficients for this scheme belong to a ray: 

=-r (j = i,..-,A), (9) 

where k > is a constant and we use the standard convention that for a cycle 

Qik+l ik = Qii ik ■ 

Lemma 2. If system (QJ) has a positive equilibrium P* then for every A4 either 
all qji = qij = or the state Ai belongs to a cycle with strictly positive rate 
constants. 

Proof. Let Ai not belong to a cycle with positive constants. We say that a state 
Aj is reachable from a state A^ if there exists a non-empty chain of transitions 
with non-zero coefficients which starts at Ak and ends at Af A^ Aj. 
Let Aii be the set of states reachable from Ai and Ai\ be the set of states Ai 
is reachable from. An n A^ — because Ai does not belong to a cycle. If Ai\ 
is not empty then in equilibrium all the corresponding p* = (j € .A4 because 
there is flow from A^ to and no flow back). If Aii is not empty then in 
equilibrium p* = because there is a flow from Ai to A^ and no flow back. 
Therefore, if the equilibrium is strictly positive and Ai does not belong to a 
cycle then A^ = Aii = ^ hence, all — qtj = 0. □ 

We will use the following simple general statement: Let Q be a cone in M. m 
without straight lines, L be a linear map, L : M m — ► R fc , and Q = L(Q) be 
a cone in M fc without straight lines. Then for every extreme ray V C Q there 
exists an extreme ray W C Q such that L(W) — V. (In other words, there 
always exists an extreme ray in the prcimagc of an extreme ray.) We will apply 
this statement to Q — (P* ) (the cone of all Markov processes with the given 
equilibrium P* ) and Q = Qg (P, P* ) (the cone of the possible velocities at point 
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P for all Markov processes with the given equilibrium). The map L transforms 
the right hand side of the Kolmogorov equation (p} into its value at point P. 
This transformation "vector field i-> its value at point P" is, obviously, a linear 
map. 

Lemma 3. Any extreme ray of the cone Q^(P*) is a simple cycle with constants 

Proof. Let a non-zero Markov chain Q with coefficients gy belong to an extreme 
ray of (P* ) . Due to Lemma [5] this chain includes a simple cycle with non- 
zero coefficients, A+ x A- lk — > (fc < n, all the numbers ii, . . . , ik are 
different, <lu +1 u > f° r J = 1, • ■ • , fc, and ik+i = ii)- For sufficiently small k 
(0 < K < /to)j »j f- > (j = 1, . . . , fc). Let Q K be the same simple cycle 

with the coefficients Then for < K < k vectors Q ± Q K also represent 
Markov chains with the equilibrium P* . Obviously, Q = LStfi^i+iSrSii) i hence, 
Q should be proportional to Q K . □ 

Due to this Lemma, every Markov chain with positive equilibrium is a convex 
combination of several simple cycles with the same equilibrium. This is the global 
decomposition of a Markov chain into simple cycles. "Global" here means that 
the same decomposition is valid for all distributions. 

Now, we are in position to prove Theorem 1. 

Proof. We will prove that any extreme ray of the cone Qg (P, P* ) corresponds 
to a simple cycle of length 2 (a step): Ai ^ Aj with the rate constants ([9]) 

Ii] = p] ' = pf ' 

According to Lemma[31 it is sufficient to prove that for any simple cycle with 
equilibrium P* and rate constants (0 and for any distribution P the right hand 
side of the Kolmogorov equation ([1]) is a conic combination (a combination with 
non-negative real coefficients) of the right hand sides of this equation for simple 
cycles of length 2 at the same point P. 

Let us prove this by induction on the cycle length k. For k = 2 it is true 

(trivially). For a cycle of length fc > 2, A\ — >• A^ — ^ A±, with the rate 

constants given by © , the right hand side of equation ([XJ) is the vector v^- with 
coordinates 

K), = V f^ - % do) 

Here, without loss of generality, we take K = 1, use index j instead of ij and 
apply the standard convention regarding cyclic order. Other coordinates of Vfc 
are zeros. 

Let us decompose this into a conic combination of a vector Vk-i for a 
cycle of length fc — 1 and a vector V2 for a cycle of length 2. The flux Aj — > Aj+i 
is pj/p*. Let us find the minimum value of this flux and, for convenience, let 
us put this minimal flux in the first position by a cyclic permutation. The 
target cycle of length fc — 1 is A-i — > . . . A^ —> A-i with rate constants given by 
formula ([9]) (n — 1). We just delete the vertex with the smallest flux from the 
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initial cycle of length k. The target cycle of length 2 is A\ ^ A 2 with the rate 

constants (f9j q 2 i = -4- , <7i2 = We find the constant k from the conditions: 

Pi P2 

Vfe = Vfc_i + V2 at the point P, hence, two following reaction schemes, (a) and 
(b), should have the same velocities, dP/dt: 

(a) A k 1/ X k A 1 1 ^ 1 A 2 and (b) A k 1 ^X k A 2 ;A 1 ^ A 2 . 
From this condition, 

= (Eh. _ _ EiV 

k > because p\jp\ is the minimal value of pj/p*. Finally, v^. = v^^i + v 2 . □ 

Further, we omit the index B or DB at the cone: Qg(P P*) = Qg B (P, P*) = 
Q n (P,P*). 

It is necessary to stress that the decomposition of the right hand side of the 
Kolmogorov equation (TTJ) into a conic combination of cycles of length 2 depends 
on the ordering of the ratios pi /p* and cannot be performed for all values of P 
simultaneously. Thus, this decomposition is local. 

2.2. Quasichemical representation and the cones of possible velocities 

For systems with detailed balance, the cone of possible velocities, Qd B (P, P* ) , 
is a polyhedral cone. For a given P*, it is a piecewise constant function of P. 
The hyperplanes of the equilibria Ai -H> Aj divide the standard simplex of distri- 
butions into a finite number of polyhedra (compartments). In each compartment 
the dominant direction of every transition Ai -f-> Aj is fixed and the cone of pos- 
sible velocities is constant. Now we find that this construction provides the 
cones of possible velocities for general Markov kinetics and not only for systems 
with detailed balance. Let us describe these cones in detail. 

The construction of cones of possible velocities was described in 1979 [3(| for 
systems with detailed balance in the general setting for generalized MAL, for 
nonlinear chemical kinetics. These systems are represented by stoichiometric 
equations of the elementary reaction coupled with the reverse reactions: 

a pl Ai + ...+ a pn A n f± /3 pl Ax + ... + (3 pn A n , (11) 

where a P i, fi P i > are the stoichiometric coefficient, p is the reaction number 
(p = 1, . . . , m). The stoichiometric vector of the pth reaction is an n dimensional 
vector 7 P with coordinates 7 P i = j3 p i — a p i . 

The equilibria of the pth pair of reactions (jlip form a surface in the space 
of concentrations. The intersection of these surfaces for all p is the equilibrium 
(with detailed balance). These surfaces of the equilibria of the pairs of elemen- 
tary reactions (|11[) divide the space of concentrations into several compartments. 
In each compartment the dominant direction of each reaction (1111) is fixed and, 
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hence, the cone of possible velocities is also constant. It is a piecewise constant 
function of concentrations (for a given temperature): 



Q = cone{7 P sign(w p ) | p = 1, . . . , m} . 

For example, let us join the transitions Ai ^ Aj in pairs (say, i > j) and 
introduce the stoichiometric vectors "f lJ with coordinates: 

( 1 if k = i, 
7 «= 1 iik=j, (12) 
[ otherwise. 

Let us rewrite the Kolmogorov equation for the Markov process with detailed 
balance ([3]) in the quasichemical form: 



(13) 



Here, w*j = qijPj — <ijip\ is the equilibrium flux from A; to Aj and reverse. 
The cone of possible velocities for (fT3"|) is 





Q"(P, P*) = cone ^ 7 J 'sign * > J • (14) 



Here, we use the three- valued sign function (with values ±1 and 0). In Fig. [5J 
the partition of the standard distribution simplex into compartments, and the 
cones (angles) of possible velocities are presented for the Markov chains with 
three states. 

A set of distributions U is positively invariant with respect to system ([T]) if 
for any initial distribution P(0) G U, the solution of (fTJ) P(t) remains in U for 
t > 0. The bold broken lines in Fig. [5] follow along the extreme rays of the angles 
of possible velocities (clockwise or anticlockwise) . They form the borders of a 
positively-invariant area for all the Markov chains with the given equilibrium 

These borders give, for example, a simple estimate of the logarithmic decre- 
ment for Markov chains. For decaying oscillations, the logarithmic decrement is 
the natural logarithm of the ratio of any two successive amplitudes: 5 = In 
For a complex eigenvalue A, the period between two amplitudes T = 27r/|3A| 
and 5 = 2tt j§xj" - For systems with detailed balance, eigenvalues are always 
real but for the general Markov chains they may be complex. For example, for 
the simple cycle A\ — > Ai — > A3 — > A\ with the equilibrium equidistribution 
Pi 2 3 = 1/3; and the rate coefficients k, the nonzero eigenvalues of the linear 
system (TTJ) are A = k(-| ± i^) and S = 2n\/3. 

Let us follow the clockwise border trajectory (Fig. [2]) starting from the state 
A\ (the corresponding distribution is P = (1,0,0)). This state belongs to the 
line of equilibria of the transition A2 ^ ^3. The first step is the equilibration 
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Figure 2: Partition of the distribution triangle for the Markov chains with three states by 
the lines -j — £1 = into twelve compartments. The corresponding cones (angles) of possi- 

ble velocities are presented. The clockwise and anticlockwise borders of the trajectories are 
represented by bold lines. 

of the transition A\ ^± Ai (A3 does not change). After that, the equilibration 
of the transition A\ f± A3 follows (A2 does not change): 



As the result of this sequence of equilibrations, when the clockwise border line 
again approaches the equilibrium line of the transition A2 ^ A3, the value of 
Pi i s (i-p^i-p*) • After this turn in angle 7r every trajectory becomes closer to 
P* . The contraction coefficient is -p. — , w, 1 ** 2 ^/, — sty or less. The anticlockwise 
trajectory gives the same contraction. We estimated the logarithmic decrement 
from below: 



2.3. Two H -theorems 

The most gen eral form of the if-theorem for Markov processes was proposed 
by Morimoto [36j. He used the following H- functions: for each convex function 
of the positive convex variable h(x) the h- divergence between distributions P 




(15) 




(16) 



and P* is 




(17) 
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At the same time these divergencies were studied by Csiszar [37[ and some- 
times they are called the Csiszar-Morimoto divergences. These functions were 
introduced two years earlier by Renyi on the last page of his famous work [38| 
together with the hint about the if-theorem. For more details see (39j . 

The time derivative of the Csiszar-Morimoto function Hh{P\\P*) (fTT)) with 
respect to master equation for a general Markov process is 



dH h {P\\P* 
dt 



Pi 




(18) 



< 



For a Markov process with detailed balance we use the quasichemical form of 
master equation (|13[) and find immediately 



dH h (P\\P*) 
dt 



MPs 



i>3 




(19) 



< 0. 



The inequality for the general Markov processes (jl8f follows from Jensen's in- 
equality in the differential form, h'(x)(y — x) < h(y) — h(x). It is valid for left 
and right limits of h' at any point x > 0. The inequality for systems with de- 
tailed balance (fl9|) follows from the monotonicity of hi . In full agreement with 
CorollaryQ] the divergences Hh(P\\P*) (|17|) are Lyapunov functions for systems 
with detailed balance and for all the Markov processes as well. Theorem Q] has 
an even stronger corollary. 

Corollary 2. For every Markov process Q with positive equilibrium P* and for 
a distribution P P* there exists a Markov process Qdb with the same equi- 
librium that obeys the detailed balance condition and has the following property: 
For every convex function h the time derivative dHh(P\\P*)/dt for Q coincides 
at point P with the time derivative of Hh(P\\P*) at this point for Qdb- 



3. Nonlinear kinetics: detailed balance versus semi-detailed balance 

The general equations of MAL without any restriction on the reaction rate 
constants demonstrate all types of non-trivial dynamic behavior, from multiple 
steady states to strange attractors 0, [4l[ . It is not a surprise because the MAL 
systems can approximate with arbitrary accuracy any smooth vector field which 



preserves the linear conservation laws and positivity of concentrations 42|, |43| . 

The systems with semi-detailed balance give the direct nonlinear generaliza- 
tion of the general Markov kinetics. They were introduced by Boltzmann for 
gas kinetics Q and generalized later for MAL systems 0, @, Q . The systems 
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with semi-detailed balance are the generalized MAL systems with additional 
relations between rate constants. 

To produce these relations, let us follow the classical work [5j and assume 
that behind the reaction mechanism (1111) there is the reaction mechanism with 
intermediate compounds B^ illustrated by Fig. [TJ Each compound is associated 
with a formal input or output complex J2i ctpiM or J2i PpiAi. Such a complex 
may participate in several reactions. Let there be k different vectors among 
{a p ,/3 p } {a p = (a pi ) and (3 p — {fi P i))- We denote these different vectors by 
Uj (j = l,...,k). The correspondent complexes are O., = Y], PjjAj. The 
reaction mechanism (fTTl) takes the form of the list of transitions Qj — » 0/ and 
the extended reaction mechanism is the list of transitions 

Qj *± B 3 -^B^Bl (20) 

Stueckelberg introduced this representation for the collisions in Boltzmann's 
equations and used two asymptotic assumptions: 

• The compounds Bj are in fast equilibrium with the corresponding input 
or output reagents and the reactions Qj ^ Bj in (j2T)]) are always close to 
equilibrium (this is the quasiequilibrium assumption, QE); 

• They exist in very small concentrations compared to other components 
(this leads to the quasi steady state approximation, QSS). 

We call the intermediates Bj compounds following the classical work of Michaelis 
and Menten j44|. In 1913, they introduced the same asymptotic assumptions 
and representation for an enzyme reaction and demonstrated that in this case 
the overall catalytic reaction obeys the MAL. 

In more general settings, these two assumption, QE and QSS, allow us to 
produce the reaction rates for the rates of the overall reactions in the form of 
the generalized MAL. The rate of the reaction 

i i 

is the product of two factors, a standard Boltzmann factor W p and a kinetic 
factor ip p > 0: 

t p = ^ p Wp = ^ p exp , (21) 

where is the chemical potential of the component A*. The corresponding 
kinetic equation is 

^ = ^E r P%" (7 P i = A»-<v)- (22) 
p 

Here TV is the vector of composition (iVj is the amount of Ai), and V is the 
volume. 
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We use the notation Cj for the concentration of Ai , c is the vector of concen- 
trations, <Tj is the concentration of Bj. The chemical potentials /ij of the compo- 
nents Ai are the partial derivatives of the free energy density, pn = df(c, T)/dci. 
The standard thermodynamic assumption about strong convexity of the func- 
tion /(c, T) for all T is accepted. 

Let us demonstrate how the generalized MAL follows from the QE and QSS 
approximations (for more details see Ref. Q). The thermodynamic equilibria 
for the extended mixture are defined as the conditional minima of the free energy 
F. The free energy of a mixture of Ai with small admixtures of the compounds 
Bj is: 

F^VHc^ + VRT^^^+^-l] . (23) 

The entropic terms VRT^j In o in this expression corresponds to the ideal gas 
equations pj — ^jRT for the partial pressure of the small admixtures of the 
compounds Bj. This ideal gas low may be valid not only in gases but for 
osmotic pressure of small admixtures in solutions (the Morse equation). 

The thermodynamic equilibria of Bj are: <;° q = <,* /Z, where Z is a positive 
number and is the standard equilibrium: 

^( C ,T)=exp(-^T) . (24) 

The thermodynamic equilibrium condition of the reactions Qj ^ Bj under the 
condition of smallness of % (QE+QSS) can be solved explicitly: 

^ = ^( C) T)ex P ( E ^]g, (c ' T) ) . (25) 

The smallness of the concentration of the compounds implies that the rates 
of the reactions Bi — > Bj in the extended mechanism (|20[) are linear functions 
of their concentrations. Let the rate constants for this first order kinetics be qji. 

In the selected approximations the extended reaction mechanism (|20l) returns 
to the form Qj —> 0j. The reaction rate of the transition Qj —> 9; in the 
quasiequilibrium approximation is rij = qijsj° ■ This is exactly the generalized 
MAL (HU) with 

At the equilibrium <;* /Z, the first order kinetics of compounds should satisfy 
the general balance condition ([2|): 

j j 

Therefore, the kinetic factors ip p satisfy the identity of semi-detailed balance: 

Y Vp = Y ^p (26) 

p,a„=u P,P p =v 
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for any vector v from the set of all vectors {ct p ,[3 p }. This identity is exactly 
the Markov balance condition ((2|) for kinetics of compounds with equilibrium 
S* /Z. It has a very transparent sense: the thermodynamic equilibrium is, at the 
same time, the equilibrium for the first order kinetics of compounds, i.e. the it 
satisfies the balance condition @ for master equation. 

Let us assume that the Markov kinetics of compounds satisfies the detailed 
balance condition ([3]) at the thermodynamic equilibrium: 

Qljtf = QjXi ■ 

Then the kinetic factors <p p satisfy the condition of detailed balance: 

where tpj is the kinetic factor for the direct reaction and ip~ is the kinetic factor 
for the reverse reaction. 

This detailed balance condition assumes that the sums in the left and right 
hand sides of Eq. (|26[) are equal term by term. Therefore, it is stronger than 
the semi-detailed balance condition. 

For linear systems, the semi-detailed balance condition turns into the stan- 
dard balance condition @ and the detailed balance condition (|27p turns into 
([3]). Of course, the class of systems with semi-detailed balance is much wider 
than the class of systems with detailed balance. Nevertheless, locally they coin- 
cide: for given thermodynamic functions i23\) and any given concentrations and 
temperature, the cone of possible velocities for systems 122]) with semi-detailed 
balance coincides with the cone of the possible velocities for the systems with 
detailed balance. 

Indeed, for the given values of concentrations we can perform the following 
three operations, (i) return from the generalized MAL to the first order kinetic 
equations of compounds, (ii) use Theorem [1] and find the system of compounds 
with detailed balance, which has the same velocity at the same point, and (iii) 
return back, to the generalized MAL. As a result, we get a kinetic system for 
the components Ai with detailed balance, the same free energy Vf(c,T) ([23]) , 
and the same velocity at the selected values of concentrations. 

4. Conclusion 

The definition of detailed balance includes the rates of all transitions at equi- 
librium but observability of all these rates together is a very special situation. 
Typically, one can observe the overall system velocity, dP/dt, or just some com- 
ponents of this velocity but not the rates of individual transitions. According to 
our results, if we know the equilibrium distribution P* and observe the system 
velocity at one nonequilibrium point P then we can never distinguish a general 
system from the systems with detailed balance. This is true for Markov kinetics 
as well as for the systems with the generalized MAL; detailed balance can never 
be distinguished from the semi-detailed balance if we know the equilibrium and 
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observe the velocity at one nonequilibrium point. The difference between ve- 
locities of the general kinetic systems and the systems with detailed balance is 
hidden in the correlations between different nonequilibrium states (or, for ex- 
ample, in the continuous pieces of trajectories). The cone of possible velocities 
at a nonequilibrium state P is a piece-wise constant function of P, which can 
be constructed explicitly for the systems with detailed balance (Fig. [5]), and the 
same construction is valid for the general kinetics. These results seem to be 
rather surprising. 

For the nonlinear mass action systems, the systems with semi-detailed bal- 
ance give the proper analogue of the general Markov kinetics. The conditions of 
semi-detailed balance were invented for the Boltzmann equation by Boltzmann 
[3] , studied by Stueckelberg [j| and rediscovered for the mass action kinetics by 
Horn and Jackson 7\. Recently [f|, it was proved that the generalized MAL 
with the semi-detailed balance condition always follows from the Markov kinet- 
ics of compounds in the Michaelis-Menten-Stueckelberg asymptotic. The class 
of the systems with semi-detailed balance is wider than the class of systems 
with detailed balance. Nevertheless, for a given equilibrium and for any given 
value of concentration these two classes have the same sets of possible velocities 
in the distribution space. 
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